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Abstract. An algorithm is described for removing extended interferences, for instance from a 
radar, which are shorter than the time of passage of a radio source across the beam of the radio 
telescope. The algorithm is developed on the basis of robust procedures with the application of 
extreme statistics. A procedure of removing "jumps" in observational records is also described. 
The results of performance of the algorithms are illustrated in the figures. The cleaning procedures 
are included in the standard data reduction system of RATAN-600. 
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1. Introduction 

In the course of astrophysical observations one often 
encounters interferences that corrupt the arriving use- 
ful signal or the statistical noise estimates in observa- 
tion records. This is especially pronounced in records 
of radio astronomical observations. In the processing 
of data, a number of noises are recorded that require 
special cleaning, which is frequently done manually, 
for instance, in the standard reduction system FADPS 
at RATAN-600 (Verkhodanov et al., 1993). 

To perform the tasks of record cleaning from dis- 
continuous jamming, extensive use is made of tech- 
nical means at the detector level (see Berlin et al., 
1995). However, when working with the archival data, 
one has to come across different kinds of noises that 
disturb the initial record. Such noises may be ordi- 
nary discontinuous jamming, radar signals, and also 
various jumps caused by the instability of ADC in 
data recording. 

Recent work concerned with the development of 
the archive ODA (Kononov et al., 1998; Kononov, 
Pavlov, 1999; Kononov et al., 1999) of RATAN-600 
observational data coming from the broad-band ra- 
diometers made it possible to solve the problems 
which demand the use of old data. Among such prob- 
lems one may isolate, for instance, investigation of the 
background radiation (Parijskij, Korol'kov, 1986) and 
search for variability of radio astronomical objects. 
The processing of these data has naturally called for 
automatic search and removal of interferences that 
corrupt the signal. Some work over control of noises 
by programme means based on nonparametric eval- 
uation of the mean was done at RATAN-600 sev- 
eral years ago (Erukhimov et al., 1990; Chernenkov, 



1996; Shergin et al., 1997). One should also mention 
the programme development based on a gradient ap- 
proach of removing discontinuous jamming which was 
initiated by Bursov (1987). The work over the cre- 
ation of the cleaning procedure described below was 
started by Pavlov in 1997 (Pavlov, 1998) under the 
supervision of the former of the authors of this paper, 
when the problem of search and removal of solitary 
interference from the radar was resolved. 

It is proposed in this paper to develop an algo- 
rithm capable of removing non-single interferences on 
a record. The algorithm employs both the robust (im- 
mune to influence) estimates of the mean on the basis 
of non-parametric rank algorithms (Huber, 1981) and 
the procedures based on extreme statistics, namely, 
minimax approach (Verkhodanov, Gorokhov, 1995), 
which enable fast evaluation of statistics changes in a 
given interval. 

Below is presented a detailed description of re- 
moval of two classes of interferences: 1) radar and 
discontinuous jamming; 2) jumps in records. 

2. Algorithm of radar interference 
cleaning 

The problems presented by the signal from a radar 
(Fig. 1) are that it is not a (5-function, that is, it 
cannot be detected with the aid of the "gradient" ap- 
proach without confusion with a real source, and it 
is quasiperiodic, i.e. it is "fuzzed up" in the region of 
spatial frequencies, which hampers its removal on the 
Fourier plane. Besides, because the signal is extended, 
the clipping (namely, removal of all points exceeding 
a specified level with subsequent replacement by pix- 
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Figure 1: Exam,ple of RATAN-600 observations at 13 cm, on 28/08/1994 with the locator operated. The record 
of the Galactic Plane is covered with the "forest" of interferences (courtesy of S.A.Trushkin). 



els according to given rules) of its spatial spectrum 
will result in corruption of the signal. For this rea- 
son the authors decided to use statistical techniques 
to detect and remove interferences produced by the 
radar. The developed procedure permitted getting rid 
of simple discontinuous jamming. In principle, within 
the frame of FADPS one can establish a procedure 
of removing interferences from radars on the basis of 
the already available modules (Verkhodanov et al., 
1992). For instance, such an algorithm can be syn- 
thesized from the modules of low noise subtraction, 
subtraction of the signal of Gaussian shape with a 
size larger than the specified one, clipping the record 
at a given level, and taking the sum of the record 
made with the subtracted low-frequency noises and 
detected sources. However, because of the multiplicity 
of Gaussians being computed and also the repetition 
of vector operations of addition/subtraction increas- 
ing the calculation time of the script operation, and 
due to the ambiguities arising in superposition of a 
useful signal and a radar signal, the authors made a 
decision to abandon this approach. 

Prior to describing of the developed algorithm we 
will explain some terms being employed in this paper. 
By the window is meant the interval on the abscissa 
axis (for our data this is a time interval measured in 
seconds) inside of which a signal search is made. By 
sliding the window along the record we mean the se- 



quential filling of the array of pixels in the appropriate 
interval when the window is shifted by 1 pixel along 
the record. By the robust evaluation of the mean is 
implied a simple median of distribution of pixel val- 
ues in the given interval, while the robust dispersion is 
the dispersion estimate made via the median absolute 
deviation (MAD) under the assumption of Gaussian 
noises (in this case if /i is MAD, then (T=/x/0.674 in 
accordance with the definition of second-order mo- 
ments for the normal distribution). 

When developing the algorithm, we used the 
archival data (Kononov et al., 1999) written in the 
form of the FITS-like F format (Verkhodanov et 
al., 1993) of the data processing system FADPS at 
RATAN-600. 

The algorithm eliminating interferences from the 
radar comprises several steps: 

1. Copying of data from the F file to the working 
array and evaluation of the robust dispersion a of 
the total record. 

2. Search for minimum and maximum values in 
the window sliding along the one-dimensional array. 
The window size is chosen equal to the beam of the 
radio telescope. As the window is sliding, an array 
of maximum peak-to-peak difference is filled in the 
current interval. The maximum scatter is computed 
as the difference of the maximum and minimum 
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values of the pixels {max-min) in the current limits 
along the axis of shift. 

3. When looking for the maximum among the 
values of differences, which exceeds the level n ■ a, 
where n is the assigned level (by default 3), analysis 
is made to search for the maximum locations by way 
of simple comparing the array values. 

4. In the region where the local maximum is 
found the behaviour of the statistics of the surround- 
ing pixels is determined in case the signal does not 
fall fully within the window. With this aim in view, 
the current position of the interval with respect to 
the signal being detected is found: right and left 
of the maximum, by means of robust averaging 
of neighbouring points. Depending on the current 
vicinity (right or left) of the observed signal, the 
location of the interval in which the parameters will 
be estimated is chosen. The real maximum must 
always fall within the given interval. 

5. Once the window position has been found, sta- 
tistical estimate of principal characteristics of the de- 
tected signal is made. For this purpose the following 
formulae (Fomalont, 1989) are used: 

t 

X = i^X,/, (6) 



B=\jj{Xt-XYlu t=l,r; 

where It is the intensity value of tlic^ f-th point in the 
current interval, Xt is the location of this point in 
the record, X is the coordinate of the source centre 
of gravity, / is the integral intensity of a possible 
sought-for signal, B is the object size, r is the size of 
the surveyed interval in pixels. 

6. Comparison of the found parameters with 
the specified ones to search for an interfering signal. 
If the determined amplitude is smaller than the 
specified one or the size (duration) of the signal 
exceeds the given one (i.e. a real source is detected), 
the window is then shifted by one pixel and a return 
to the point of calculation of {max-min) statistics is 
accomplished. 

7. If the parameters correspond to the expected 
properties of an interference, the boundaries of the 
signal to be removed are defined. 

8. In the procedure of replacement of the points 
of the record within the limits found, provision is 



made for variants of replacing the generated noise 
with characteristic properties (dispersion and mean) 
of the noise process of the present record or ordinary 
regression by two estimated means at the edges of 
the record bordering on the current interval. To gen- 
erate a noise Gaussian process, the standard library 
function of the language "C" rand() is used, which 
outputs uniformly distributed values. For conversion 
to the Gaussian noise, we applied a simple procedure 
of averaging with a shifted mean and normalizing: 

m 

Ni = a\/l2m — , 

2m 

where R is the vector of random numbers of the uni- 
form distribution of the fimction rand() returned by 
the generator, m is the number of the points involved 
in averaging of uniformly distributed numbers. The 
larger m, the better the distribution corresponds to 
normal. We employed to=32 in our procedure. The 
value of the current system time is used as the origin 
of the generator. 

9. If there is one more maximum in a specified 

interval, then return to the point of Hc;arch for the 
location of the maximum in the current window (see 
above) is performed. 

10. If the end of the record is not reached, a shift 
is made by the size of the interval of cleaning along 
the axis. 

11. When the record is finished the data are out- 
put to the F file. 

The schematic of the radar interference removal 
algorithm described above is displayed in Fig. 2. 

The results of operation of the algorithm pre- 
sented in Fig. 3. 

Fig. 4 shows the power Fourier spectra of the two 
records displayed in Fig. 3: prior to cleaning and after 
it. To highlight the diff'erence, one and the same curve 
of the low-frequency background is subtracted from 
both records. The difference between the spectra is 
caused by elimination of the discontinuous jamming 
and quasiperiodic radar signal. 

3. Algorithms of removing "jumps" 

Apart from interferences treated by the described al- 
gorithm, there is a class of interferences not infre- 
quently found in archival records: jumps of the mean 
or "steps" (Fig. 6). We have developed a procedure of 
search for the jump on the basis of analysis of the 
mean over four neighbouring sliding intervals. 
The algorithm is described as follows: 
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Figure 3: Results of clipping interferences in a record of wide-band radiometers. The upper panel shows a record 
with a source and impulse, and locator interferences. The lower panel shows a record cleaned with the algorithm 
of extended interference clipping with noise addition. 
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Figure 4: Power Fourier spectrum of records before (a) and after (b) cleaning (see Fig. 3). The difference is 
due to cleaning of two impulse interferences and a quasiperiodic signal of the locator. 
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Figure 6: Results of removing jumps in a record. The upper panel shows a record with a jump. The lower panel 
shows a cleaned record. 



1. Reading data from the F file to the working 
array and estimation of the robust dispersion a of 
the total record. 

2. Fining the array M„, where every element is 
an estimate of the mean in the sliding window of a 
specified size. It is desirable that the size of the win- 
dow should be chosen not shorter than 5 pixels. 

3. Location of the interval in which a jump oc- 
curred. Here we analyse the array of the mean values 
estimated by a robust technique with the aim of re- 
vealing the position of the abrupt increase of the mean 
(> na, by default n=3) value (gradient approach) by 
way of comparing the values of four elements of the 
array M . The jump is considered to be detected in 
the i-th interval provided the following conditions are 
satisfied at the same time: 



< 



|Mi_i -M,+i| < na, 



\M, 



i+l 



M, 



i+2 



< a. 



Thus the scheme of coincidence of the mean in two 
neighbouring intervals at the edges and of uneven 
transition in the central window is realized. 

4. Revision of the jump location (for instance, on 
the fc-th pixel) by means of comparison. 

5. Decrease of the values of all pixels following 
the fc-th pixel by the value of the jump amplitude. 



6. Decrease of all values in the array of the means 
in the sliding window, which follow the current value 
of the mean. 

7. If there is no limitation on the length of the 
record, a shift along the record and jump to the condi- 
tion of the check point are performed. In other words, 
the cycle is completed. 

8. Recording to the F file. 

In Fig. 5 is shown a schematic diagram of the 
above-described algorithm of searching for and re- 
moving the jump in the records. The results of per- 
formance of the algorithm are demonstrated in Fig. 6. 

4. Conclusions 

Through analysing statistical properties in the cur- 
rent interval of the record the developed algorithms 
find and remove interferences of different kinds 
quickly and effectively. 

The procedure is involved as a command "clip" 
and included in the standard data processing system 
FADPS at RATAN-600. 

Provision is made for possible extension of the al- 
gorithm to two-dimensional data and its application 
to removal of cosmic particle signals on CCD images. 
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